Probabilistic Acceptors for Languages over Infinite Words

Probabilistic ω-automata are variants of nondeterministic automata for infinite words where all choices are resolved by probabilistic distributions. Acceptance of an infinite input word requires that the probability for the accepting runs is positive. In this paper, we provide a summary of the fundamental properties of probabilistic ω-automata concerning expressiveness, efficiency, compositionality and decision problems. While classical finite automata can serve to recognize languages over finite words or tree-like structures, ω-automata are acceptors for languages consisting of infinite objects. ω-automata play a central role for verification purposes, reasoning about infinite games and logics that specify nondeterministic behaviors. Many variants of ω-automata have been studied in the literature that can be classified according to their inputs (e.g., words or trees), their acceptance conditions (e.g., Büchi, Rabin, Streett, Muller or parity acceptance) and their semantics of the branching structure (e.g., deterministic, nondeterministic, or alternating). See, e.g., [Tho97, GTW02] for an overview of automata over infinite objects. In this paper, we study probabilistic variants of ω-automata for languages over infinite words. Although probabilistic finite automata (PFA) have attracted many researchers, see e.g. [Rab63, Paz66, Fre81, MHC03, DS90, BC03], probabilistic language acceptors for infinite words just have recently been studied. The formal definition of probabilistic ω-automata is roughly the same as for nondeterministic ω-automata, except that all choices are resolved by probabilistic distributions. Acceptance of an infinite word σ = a1 a2 a3 . . . requires that the generated sample run for σ (i.e., sequence of states that are passed in the automaton while reading σ letter by letter) meets the acceptance condition with positive probability. For instance, in the case of a probabilistic Büchi automaton (PBA), certain states are declared to be accepting and the acceptance condition requires to visit some accepting state infinitely often with positive probability. As this definition of the accepted language via the criterion “the probability for the accepting runs is > 0” appears to be the natural adaption of the definition of the accepted language of a nondeterministic automaton which relies on the criterion “there is at least one accepting run”, one might expect that probabilistic and nondeterministic ω-automata are rather close and enjoy similar properties. This, however, is not the case. The first surprising result is that PBA are more expressive than nondeterministic Büchi automata (NBA). Second, concerning the sizes of smallest automata for a given language, probabilistic and nondeterministic ω-automata are not comparable. That is, M. Nielsen et al. (Eds.): SOFSEM 2009, LNCS 5404, pp. 19–33, 2009. c © Springer-Verlag Berlin Heidelberg 2009 20 C. Baier, N. Bertrand, and M. Größer there are languages that can be accepted by PBA of polynomial size, while all NBA for these languages have at least exponentially many states, and vice versa. Another interesting observation is that in the probabilistic setting the Büchi condition is somehow more powerful than in the nondeterministic case, as there exists a polynomial transformation from PBA to probabilistic automata with Streett acceptance. This is known to be impossible in the nondeterministic case [SV89]. On the other hand, the price we have to pay for this extra power of PBA is that we lose decidability of algorithmic problems, such as the emptiness, universality or equivalence problem. The undecidability results for PBA have several important consequences. First, the concept of PBA is not adequate for solving algorithmic problems that are related to the emptiness or universality problems. This, e.g., applies to the verification of nondeterministic systems against PBA-specifications. Second, PBA can be viewed as a special instance of partially-observable Markov decision processes (POMDPs) which are widely used in various areas, including robotics and stochastic planning (see, e.g., [Son71, PT87, Lov91]) and the negative results established for PBA yield the undecidability of various verification problems for POMDPs. Organization. Section 1 recalls the definition of nondeterministic ω-automata with Büchi, Rabin or Streett acceptance conditions and introduces their probabilistic variants. Results on the expressiveness and efficiency of probabilistic Büchi, Rabin and Streett automata are summarized in Section 2. Composition operators for PBA are addressed in Section 3. Decision problems for PBA and the relation to POMDPs will be discussed in Section 4. Finally, Section 5 contains some concluding remarks. The material of this paper is a summary of the results presented in the papers [BG05, BBG08]. Further details can be found there and in the thesis by Marcus Größer [Grö08]. 1 From Nondeterministic to Probabilistic ω-Automata We assume some familarity with classical nondeterministic automata over finite or infinite words; see e.g. [Tho97, GTW02]. We just recall some basic concepts of nondeterministic ω-automata. Later we will adapt these concepts to the probabilistic setting. Definition 1 (Nondeterministic ω-automata). A nondeterministic ω-automaton is a tuple N = (Q,Σ,δ,Q0,Acc), where – Q is a finite nonempty set of states, – Σ is a finite nonempty input alphabet, – δ : Q×Σ → 2Q is a transition function that assigns to each state q and letter a∈ Σ a (possibly empty) set δ(q,a) of states, – Q0 ⊆ Q is the set of initial states, – Acc is an acceptance condition (which will be explained later). N is called deterministic if |Q0| = 1 and |δ(q,a)| = 1 for all q ∈ Q and a ∈ Σ. The intuitive operational behavior of a nondeterministic ω-automaton N for some infinite input word σ = a1 a2 a3 . . . ∈ Σω is as follows. The automaton selects nondeterministically an initial state q0 ∈ Q0. Then, it attempts to read the first letter a1 in Probabilistic Acceptors for Languages over Infinite Words 21 state q0. If q0 does not have an outgoing a1-transition (i.e., δ(q0,a1) = / 0) then the automaton rejects. Otherwise, the automaton reads the first letter a1 and moves to some a1-successor q1 of q0 (i.e., some state q1 ∈ δ(q0,a1)) and attempts to read the remaining word a2 a3 . . . from state q1. That is, the automaton rejects if δ(q1,a2) = / 0. Otherwise the automaton reads letter a2 and moves to some state q2 ∈ δ(q1,a2) which is chosen nondeterministically, and so on. Any maximal state-sequence π = q0 q1 q2 . . . that can be obtained in this way is called a run for σ. We write inf(π) for the set of states p ∈ Q such that p = qi for infinitely many indices i ≥ 0. Each finite run q0 q1 . . .qi (where N fails to read letter ai+1 in the last state qi because δ(qi,ai+1) is empty) is said to be rejecting. The acceptance condition Acc imposes a condition on infinite runs and declares which of the infinite runs are accepting. Several acceptance conditions are known for nondeterministic ω-automata. We will consider three types of acceptance conditions: Büchi: A Büchi acceptance condition Acc is a subset F of Q. The elements in F are called final or accepting states. An infinite run π = q0 q1 q2 . . . is called (Büchi) accepting if π visits F infinitely often, i.e., inf(π)∩F (= / 0. Streett: A Streett acceptance condition Acc is a finite set of pairs (Hl ,Kl) consisting of subsets Hl,Kl of Q, i.e., Acc = {(H1,K1), . . . ,(H!,K!)}. An infinite run π = q0 q1 q2 . . . is called (Streett) accepting if for each l ∈ {1, . . . ,!} we have: inf(π)∩Hl (= / 0 or inf(π)∩Kl = / 0. Rabin: A Rabin acceptance condition Acc is syntactically the same as a Streett acceptance condition, i.e., a finite set Acc = {(H1,K1), . . . ,(H!,K!)} where Hl,Kl ⊆ Q for 1 ≤ l ≤ !. An infinite run π = q0 q1 q2 . . . is called (Rabin) accepting if there is some l ∈ {1, . . . ,!} such that inf(π)∩Hl = / 0 and inf(π)∩Kl (= / 0. Note that the semantics of Streett and Rabin acceptance conditions are duals of each other, i.e., for each infinite run π we have: π is accepting according to the Rabin condition Acc iff π is rejecting (i.e., not accepting) according to the Streett condition Acc. Furthermore, a Büchi acceptance condition F can be viewed as a special case of a Streett and Rabin condition with a single acceptance pair, namely {(F,Q)} for the Streett condition and {( / 0,F)} for the Rabin condition. The accepted language of a nondeterministic ω-automaton N with the alphabet Σ, denoted L(N ), is defined as the set of infinite words σ ∈ Σω that have at least one accepting run in N . L(N ) = { σ ∈ Σω : there exists an accepting run for σ in N } In what follows, we write NBA to denote a nondeterministic Büchi automaton, NRA for nondeterministic Rabin automata and NSA for nondeterministic Streett automata. Similarly, the notations DBA, DRA and DSA are used to denote deterministic ω-automata with a Büchi, Rabin or Streett acceptance condition. 22 C. Baier, N. Bertrand, and M. Größer It is well-known that the classes of languages that can be accepted by NBA, DRA, NRA, DSA or NSA are the same. These languages are often calledω-regular and represented by ω-regular expressions, i.e., finite sums of expressions of the formα.βω where α and β are ordinary regular expressions (representing regular languages over finite words) and the language associated with β is nonempty and does not contain the empty word. In the sequel, we will identify ω-regular expressions with the induced ω-regular language. While deterministic ω-automata with Rabin and Streett acceptance (DRA and DSA) cover the full class of ω-regular languages, DBA are less powerful as, e.g., the language (a + b)∗aω cannot be recognized by a DBA. Hence, the class of DBA-recognizable languages is a proper subclass of the class of ω-regular languages. Probabilistic ω-automata can be viewed as nondeterministic ω-automata where all nondeterministic choices are resolved probabilistically. That is, for any state p and letter a ∈ Σ either p does not have any a-successor or there is a probability distribution for the a-successors of p. Definition 2 (Probabilistic ω-automata). A probabilistic ω-automaton is a tuple P = (Q,Σ,δ,μ0,Acc), where – Q is a finite nonempty set of states, – Σ is a finite nonempty input alphabet, – δ : Q×Σ×Q→ [0,1] is a transition probability function such that for all p∈ Q and a ∈ Σ either ∑q∈Q δ(p,a,q) = 1 or δ(p,a, .) is the null-function (i.e. δ(p,a,q) = 0 for all q ∈ Q), – μ0 is the initial distribution, i.e., a function μ0 : Q → [0,1] such that ∑q∈Q μ0(q) = 1, – Acc is an acceptance condition (as for nondeterministic ω-automata). We refer to the states q0 ∈ Q where μ0(q0) > 0 as initial states. If p is a state such that δ(q,a, p) > 0 then we say that q has an outgoing a-transition to state p. Acceptance conditions can be defined as in the nondeterministic case. In this paper, we just regard Büchi, Rabin and Streett acceptance and use the abbreviations PBA, PRA and PSA for probabilistic Büchi automata, probabilistic Rabin automata, and probabilistic Streett automata, respectively. The intuitive operational behavior of a probabilistic ω-automaton P for a given input word σ = a1a2 . . . ∈ Σω is similar to the nondeterministic setting, except that the nondeterminism is resolved internally by the probabilistic distributions μ0 in the initial configuration and δ(q,a, ·) if the current state is q and the next letter to consume is a. That is, initially P chooses at random an initial state p0 according to the initial distribution μ0. If P has consumed the first i input symbols a1, . . . ,ai and the current state is pi then P moves with probability δ(pi,ai+1, p) to state p and tries to read the next input symbol ai+2 in state p = pi+1. If there is no outgoing ai+1-transition from the current state pi, then P rejects. As in the nondeterministic case, the resulting state-sequence π = p0 p1 p2 . . . ∈ Q∗ ∪Qω is called a run for σ in P . If P rejects in state pi, i.e., δ(pi,ai+1, ·) is the null function, then the obtained run is finite (and ends in state pi). If the automaton never rejects while reading the letters ai of the input word σ = a1a2a3 . . . ∈ Σω, the generated Probabilistic Acceptors for Languages over Infinite Words 23 run is an infinite state-sequence π = p0 p1 p2 . . . ∈ Qω. Acceptance of a run according to a Büchi, Rabin or Streett acceptance condition is defined as in the nondeterministic setting. Semantics of probabilistic ω-automata. While acceptance of an infinite word in a nondeterministic ω-automata requires the existence of an accepting run, a probabilistic ωautomaton accepts an infinite input word σ if the acceptance probability PrP (σ) is > 0. The formal definition of the acceptance probability relies on the view of an input word σ ∈ Σω as a scheduler when P is treated as a Markov decision process, i.e., an operational model for a probabilistic system where in each state q the letters that can be consumed in q are treated as actions that are enabled in q. Given a word/scheduler σ = a1 a2 a3 . . . ∈ Σω, the behavior of P under σ is given by a Markov chain Mσ where the states are pairs (q, i) where q∈Q stands for the current state and i is a natural number ≥ 1 that denotes the current word position. Stated differently, state (q, i) in the Markov chain Mσ stands for the configuration that P might have reached after having consumed the first i−1 letters a1, . . . ,ai−1 of the input word σ. Assuming that δ(q,ai+1, ·) is not the null function, the transition probabilities from state (q, i) are given by the distribution δ(q,ai+1, ·), i.e., from state (q, i) the Markov chain Mσ moves with probability δ(q,ai+1, p) to state (p, i + 1). In case that δ(q,ai+1, ·) = 0 then (q, i) is an absorbing state, i.e., a state without any outgoing transition. The runs for σ in P correspond to the paths in Mσ. We can now apply the standard concepts for Markov chains to reason about the probabilities of infinite paths and define the acceptance probability for the infinite word σ in P , denoted PrP (σ) or briefly Pr(σ), as the probability measure of the accepting runs for σ in the Markov chain Mσ. The formal definition of the accepted language of P is L(P ) = { σ ∈ Σω : PrP (σ) > 0 } Sometimes we will add the subscript “Büchi”, “Streett’ or “Rabin” to make clear which type of acceptance condition is assumed and write LBüchi(P ), LRabin(P ) or LStreett(P ), respectively. Example 1 (Probabilistic Büchi automata). Let us have a look at a few examples of probabilistic ω-automata. In the pictures, if δ(q,a, ·) (= 0 then the probability δ(q,a, p) is attached to the a-transition from q to p. If δ(q,a, p)= 1 then the edge is simply labeled with a. Similarly, if there is a single initial state q0 (i.e., μ0(q0) = 1, while μ0(q) = 0 for all other states q) we simply depict an incoming arrow to q0. For PBA, we depict the accepting states (i.e., the states q ∈ F) by squares, non-accepting states by circles. Consider the PBA P over the alphabet Σ = {a,b} in the left part of Figure 1. State q0 is initial, while state q1 is accepting. More precisely, the initial distribution is given by μ0(q0) = 1 and μ0(q1) = 0, while the Büchi acceptance condition is given by the singleton F = {q1}. The accepted language L(P ) = LBüchi(P ) is (a+b)∗aω. If we feed P with an infinite input word σ ∈ (a + b)∗aω, then P stays with positive probability in the initial state q0 until the last b in σ has been read. From then on, P moves almost surely to the accepting state q1 and stays there forever when reading the infinite suffix aω. Thus, the acceptance probability for all words in (a + b)∗aω is positive. This yields that (a + b)∗aω ⊆ L(P ). 24 C. Baier, N. Bertrand, and M. Größer